Skip to content

Conversation

@jorgebaralt
Copy link
Contributor

Add Usage Tracking & Aggregation System

Overview

Implements comprehensive usage tracking for AI model consumption with cost calculation, quota management, and webhook-based processing.

Changes

Database Schema

  • New tables: usage_event, usage_period, usage_aggregate, model_pricing
  • Indexes: Added performance indexes for user queries and date ranges
  • Relations: Proper foreign keys to users, plans, and subscriptions

API Endpoints

  • api.usage.checkUsage - Check user quota against current usage
  • api.usage.trackModelUsage - Record AI model usage events
  • api.usage.processUsageEvent - Process events and calculate costs
  • api.modelPricing.* - Manage model pricing data

AI Integration

  • Journl Agent: Track usage after streaming completion (input/output/reasoning tokens)
  • Blocknote: Real-time usage tracking during streaming
  • Webhook Processing: Asynchronous cost calculation via Supabase webhooks

Key Features

  • Cost Calculation: Historical pricing lookup for accurate billing
  • Quota Management: Real-time usage checking (free vs pro plans)
  • Idempotency: Prevents duplicate processing of webhook events
  • Transaction Safety: Atomic operations with proper rollback handling

Infrastructure

  • New Supabase webhook endpoint: /api/supabase/usage

Next Steps

  • Create a new procedure to check usage
  • add the new checkUsage in the correct places to prevent/allow usage if needed

@jorgebaralt jorgebaralt marked this pull request as ready for review October 4, 2025 21:15
@jorgebaralt jorgebaralt requested a review from rmolinamir October 4, 2025 21:15
Comment on lines 27 to 53
const activeSubscription = await getActiveSubscription({
ctx,
userId,
});

if (activeSubscription) {
// Validate subscription has required fields
if (
!activeSubscription.periodStart ||
!activeSubscription.periodEnd ||
!activeSubscription.plan
) {
throw new TRPCError({
code: "INTERNAL_SERVER_ERROR",
message: "Invalid subscription data",
});
}

// Pro user: try to find existing usage period for current subscription period
let usagePeriod = await tx.query.UsagePeriod.findFirst({
where: and(
eq(UsagePeriod.user_id, userId),
eq(UsagePeriod.subscription_id, activeSubscription.id),
eq(UsagePeriod.period_start, activeSubscription.periodStart),
eq(UsagePeriod.period_end, activeSubscription.periodEnd),
),
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should get both the activeSubscription and the usagePeriod in 1 query; otherwise, we are doing two round-trips to the DB, which is a lot of lag. We'll notice things are slower if we use this everywhere. I think we can get both at the same time with Drizzle relationships - worst case scenario we can write a SQL query lol.

Comment on lines 94 to 127
// Try to find existing usage period for current month
let usagePeriod = await tx.query.UsagePeriod.findFirst({
where: and(
eq(UsagePeriod.user_id, userId),
eq(UsagePeriod.period_start, monthStart),
eq(UsagePeriod.period_end, monthEnd),
),
});

if (!usagePeriod) {
// Get the free plan for free users
const freePlan = await getFreePlan(ctx);

// Create usage period for free user
const [newUsagePeriod] = await tx
.insert(UsagePeriod)
.values({
period_end: monthEnd,
period_start: monthStart,
plan_id: freePlan?.id,
subscription_id: null, // Free users don't have a subscription_id
user_id: userId,
})
.returning();

usagePeriod = newUsagePeriod;
}

if (!usagePeriod) {
throw new TRPCError({
code: "INTERNAL_SERVER_ERROR",
message: "Failed to create or retrieve usage period for free user",
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like doing a write here since this is supposed to be a query; if we combine reads with writes here, it will be very hard to optimize later. This is meant to be a middleware being used everywhere 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe:

  • When users create accounts, create a usage period too
  • When users subscribe, update the period?
  • When user subscriptions end, create a new period?
  • Then we could reset usage in a cron job for free users 🤔

This way we separate writes/reads.

recurring: insertData.recurring,
type: insertData.type,
unitAmount: insertData.unitAmount,
updatedAt: new Date(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Comment on lines 26 to 29
created_at: timestamp().defaultNow(),
updated_at: timestamp()
.defaultNow()
.$onUpdateFn(() => new Date()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should look like this:

Suggested change
created_at: timestamp().defaultNow(),
updated_at: timestamp()
.defaultNow()
.$onUpdateFn(() => new Date()),
created_at: t
.timestamp({ mode: "string", withTimezone: true })
.defaultNow()
.notNull(),
updated_at: t
.timestamp({ mode: "string", withTimezone: true })
.defaultNow()
.notNull()
.$onUpdateFn(() => sql`now()`),

Comment on lines 30 to 33
created_at: timestamp().defaultNow(),
updated_at: timestamp()
.defaultNow()
.$onUpdateFn(() => new Date()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same date-related changes here.

usage_period_id: t
.uuid()
.notNull()
.references(() => UsagePeriod.id, { onDelete: "cascade" }),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should ever delete usage when periods are deleted for tracking purposes 🤔

We should probably never delete periods either.

Comment on lines 18 to 21
created_at: timestamp().defaultNow(),
updated_at: timestamp()
.defaultNow()
.$onUpdateFn(() => new Date()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same date related changes could be done here.

- free user: created on create account, new periods created via cronjob
- paid user: created once user subscribes
@rmolinamir rmolinamir merged commit 59f0f9d into main Oct 24, 2025
3 checks passed
@rmolinamir rmolinamir deleted the jorge/usage-aggregate branch October 24, 2025 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants